Three-dimensional computer animation continues to intensely impact the visual arts world. In the entertainment industry, for example, three-dimensional computer animation has provided the sole basis for several motion pictures, and computer-generated visual effects are increasingly used to enhance or replace filmed scenery, special effects, and stunts in filmed motion pictures. Additionally, three-dimensional computer animation has furthered the sciences and practical arts by allowing, for example, visual analysis of product designs prior to physical assembly (e.g., aircraft designs), and lifelike visualizations of planned events (e.g., spacecraft landings).
As described in Kerlow, The Art of 3-D Computer Animation and Imaging, Wiley & Sons, Inc. (2000), the production stage in the process of three-dimensional computer animation includes the steps of modeling, animation, and rendering. These tasks usually take place in a computer animation studio in a collaborative effort among many creative, technical, production, and/or administrative personnel (hereinafter “users”), although there may be only a single user for the simplest endeavors. The computer animation studio may comprise one physical site or may be distributed across several sites, in which case the sites are usually connected by a wide area network.
Modeling refers to the creation of the virtual characters, objects, and environments that will be used in the movie or feature being produced. Computer modeling can occur using a wide range of computer based three-dimensional techniques, ranging from virtual modeling techniques for the direct creation of virtual objects, to the use of three-dimensional digitizers to capture the shape of real-world objects and form virtual objects therefrom. For each object, the modeling process yields a body of numerical and symbolic information (hereinafter “model”) that describes its geometry and other characteristics, usually in isolation from other objects. Models of different but related objects (e.g., hat, arm, spurs) can be grouped together in defining a model of a larger object (e.g., cowboy) as needed. From a computing perspective, an object model comprises a file or array comprising numerical and symbolic information sufficient to define the object according to a high-level modeling language. Examples of high-level modeling languages include VRML (Virtual Reality Modeling Language), Lightwave, Softimage, Maya, and 3D Max formats. The size of the file or array containing the model will increase, of course, as the complexity of the model increases. Once the virtual actors and objects are modeled, they can be arranged on a virtual stage and animated.
Animation refers to the incremental changing of the positions and orientations of models in three-dimensional space, such that samples of the models and their arrangement on the virtual stage can yield an illusion of continuous action when viewed in succession. Computer animation techniques range from keyframing animation in which starting and ending positions are specified for all objects in a sequence, to motion capture in which all positions are fed to the objects directly from live actors whose motions are being digitized.
The animation process results in a sequence of logically related samples that together form a “shot,” with a collection of logically related shots forming a “scene” of an overall production. As known in the art, the term “scene” additionally refers to the selection and configuration of the virtual objects, characters, and environments that are the subject of the sequence being animated. Thus, for example, a computer-animated feature film may contain a scene of a wild West gunfight, the scene comprising virtual models of two cowboys, their horses, and a crowd positioned against a virtual town environment, the scene further comprising a first shot of a close-up of one cowboy as he warns the other cowboy, a second shot of the cowboys pointing their guns at each other, a third shot of the crowd gasping, and so on.
From a computing perspective, the result of the animation process is a sequence of three-dimensional representations of object models in progressively different positions and orientations, each member usually being expressed according the same high-level modeling language supra. At this point, the virtual objects are still expressed in three-dimensional form and are not suitable for viewing, except perhaps in “wireframe” renditions as the animation is perfected. Once the virtual objects are modeled and animated, they can then be rendered.
Rendering refers to the process of visually representing the animated models with the aid of a simulated camera, thereby producing the output images that are actually viewed by the audience. In many ways, the rendering process is analogous to the cinematography process for standard filmed movies. A Hollywood cinematographer must optimally capture physical actors and objects on a movie stage with a properly positioned camera, carefully selected lighting, and other carefully selected parameters for transferring the scene onto a tangible two-dimensional viewable medium (film). Likewise, the computer rendering process must optimally capture virtual actors and virtual objects on a virtual three-dimensional stage, capturing them with a properly positioned simulated camera, carefully selected virtual lighting, and other carefully selected parameters for transferring the virtual scene onto a tangible two-dimensional viewable medium (a digital image file). The output of the computer rendering process is a sequence of two-dimensional images that form the final product viewable by an audience. The lighting of the scene and the shading characteristics are often specified before the animation is laid out, but the rendering itself, the calculation of the finished images, necessarily happens after the modeling and animation parameters have been defined.
The production-stage tasks of modeling, animation, and rendering can be performed using any of a variety of application software packages ranging from proprietary software solutions to commercial off-the-shelf software packages. Because there is tremendous latitude in the way production-stage tasks might be achieved, different commercial software packages have arisen and/or evolved that perform different combinations and sub-combinations of the above production-stage tasks. Thus, some software packages may be for modeling only or animation only, other packages may perform both modeling and animation, still other packages may perform rendering only, and still other packages may perform all three functions.
Additionally, many adaptors and interfaces exist for allowing, for example the rendering engine of a first package to render the modeled and animated data generated by a second package, commonly through a “plug-in” that allows the user to remain in the environment of the second package while manipulating the rendering data and rendering controls. By way of example and not by way of limitation, RenderMan™ is a popular collection of rendering tools available from Pixar Animation Studios, Inc. of Emeryville, Calif., that includes a rendering program, a scene description interface, and a shading language. A user may choose to use Maya®, a popular modeling and animation package (also capable of rendering) available from Alias/Wavefront of Toronto, Canada, to model and animate a scene, and then use MTOR, a RenderMan plug-in developed for Maya, to connect Maya to the RenderMan engine. Maya primitives are converted into a RIB (RenderMan Interface Bytestream) file. The RIB file is then interpreted by the RenderMan engine to produce rendered frames.
It is to be appreciated that while some terminology infra may have some elements in common with the RenderMan package, the Maya package, or other commercial software packages, the descriptions of the preferred embodiments are not intended to be limited to these restricted environments. Rather, the preferred embodiments described infra may applied in many different contexts using any of a variety of modeling, animation, and/or rendering programs. The preferred embodiments described herein are generally applicable to any environment in which rendering data and rendering controls are produced by a modeling application, an animation application, or other graphics application, and are then provided to a rendering engine that produces rendered frames.
Most commonly, to carry out the rendering process, the user manipulates their modeling application (e.g., Maya, Lightwave, etc.) to generate rendering data and rendering controls. The user then instantiates a rendering process in which the rendering data and rendering controls are submitted to a rendering engine. In a process that is highly computationally intensive, the rendering engine then produces rendered frames in the form of digital image files in any of a variety of formats (e.g., jpg, gif tif, etc.). The process is highly iterative, with the rendering data and/or rendering controls being modified and tweaked until the desired output is achieved. Generally speaking, each time any portion of the rendering data or rendering controls is adjusted, the entire computing process performed by the rendering engine must be repeated.
More specifically, the user manipulates their modeling application in a first step of getting the models to be rendered from some kind of peripheral storage device like a hard disk. These models usually include virtual characters, props, sets, and other objects. In another step, a simulated camera is maneuvered in virtual x-y-z space so that the user can look at the portion of the environment they are interested in. They might reposition the camera, tilt it, change the focal point and depth of field, and adjust proportions and resolution parameters. In another step, the lighting scheme is designed, the user placing at least one light source, and often several light sources, in the three dimensional space of the computer software. In another step, the user specifies many characteristics of the surfaces of the objects including color, texture, shininess, reflectivity, and transparency. Selection of these rendering parameters will have a great impact on the quality, refinement, and energy of the rendered frames produced by the rendering engine. Finally, the user selects a shading method (e.g. faceted, smooth, specular, RenderMan, etc.). Further descriptions of shading methods and other specific rendering controls can be found in Kerlow, supra. For purposes of the present disclosure, it is of significance primarily to note that each of the many sets of user modifications/tweaks of the rendering data or the rendering controls results in the need for an additional rendering job to be submitted to the rendering engine.
After rendering controls and rendering data are specified, the user submits a rendering request to instantiate the transfer of the rendering data and the rendering controls to the rendering engine. This is usually performed by pressing a “render” control button provided by the rendering engine plug-in to the modeling application. The modeling application then provides the rendering data and rendering controls to the rendering engine.
As known in the art, when the geometry or shading in any given scene are too complex, it is common to render different components of a scene in separate layers. The rendered layers are then later composited in a post-production process. For clarity of disclosure, the term “rendered frame” is used to identify a rendered output corresponding to one time sample of a shot, scene, or a layer thereof, it being understood that layered outputs would be later composited to form the actual output image viewed by the audience. The rendered frames are usually displayed to the audience at a rate of 24 frames per second (fps) for film and 30 fps for video. A rendering session maps on to a subset of a movie production flow—be it a scene or a shot, or on to a smaller rendering task. The larger extent of the workflow, for which sessions are conducted is termed a project.
FIG. 1 shows a conceptual hierarchy of rendering data and rendering controls as provided from a modeling application/rendering engine plug-in upon instantiation of a rendering request. Rendering data 102 generally comprises appearance parameters and geometry parameters. Appearance parameters comprise shaders, provided in the form of shader files, and textures, provided in the form of texture files. As known in the art, shaders control the appearance of the site of the scene, specifying, for example, lighting and surface behaviors (e.g., matte, plastic, specular, etc.). Also as known in the art, textures are used to further specify the appearances of surfaces, and are provided in the form of 2-D image files, or alternatively as 3-D image volumes that comprising a plurality of related 2-D image files. Textures are usually very large data files, exceeding a total volume, for example, of 250 Gbytes for an entire project. There is a dependency between shaders and textures in that the shaders reference textures in mapping the textures onto objects during the rendering process.
Geometry parameters comprise procedural geometry information (“procedurals”), provided in the form of procedural geometry files (“procedural files”), as well as geometries provided by the scene description files. As known in the art, procedurals are used to describe geometric elements in terms of algorithms, rather than in terms of simple coordinates, providing a way to model more complicated objects. Scene description files describe the overall scene in terms of both scene descriptions and object geometries. They are first specified by a modeling program, supra, responsive to manipulations by a user. Thus, scene description files are considered to contain both geometry information on the rendering data side of FIG. 1, as well as scene description information on the rendering controls 104 side of FIG. 1. The scene description file is where all the geometry (polygons, surfaces), as well as shaders and procedurals, are referenced. As indicated in FIG. 1, and as used herein, rendering resources shall refer to the collection of scene description files, shader files, texture files, and procedural files used by the rendering engine in generating rendered images.
FIG. 2 shows a conceptual diagram of the “generation” process associated with rendering resources in most conventional 3-D computer animation systems today. Generally speaking, as a result of user manipulations, the modeling application will provide raw shader files 202, raw procedural files 204, and raw scene description files 208 for downstream rendering. Although they are automatically created by the modeler, these raw or “uncompiled” files are usually in ASCII format or other human-readable format and are not directly usable by the rendering engine. Rather, they must first be compiled or “generated” in a preprocessing step prior to use by the rendering engine.
As indicated in FIG. 2, generation of raw shader files 202 (provided in the form of .sl, .c, .cpp. or .h files as known in the art) is an intrinsic process that depends largely on a set of independent shader make parameters 210, also provided by the modeling application. Upon generation, generated shader files 218 (e.g., so files) can be provided to the rendering engine. Likewise, generation of raw procedural files 204 (provided in the form of .c, .cpp. or .h files as known in the art) is an intrinsic process that depends on independent procedural make parameters 214, and generation of raw scene description files 208 (e.g., provided in the form of .rib files as known in the art) is an intrinsic process that depends on independent scene description make parameters 216. Upon generation, generated procedural files 220 (e.g., so files) and generated scene description files 224 (e.g., rib files) can be provided to the rendering engine.
Unlike the other rendering resource files, texture files 206 (e.g., tif, jpg, gif, etc) may be created separately from the modeling application, e.g., a user may have get them from digital camera outputs, artist drawings, and the like. However, texture files 206 may also be modified or provided by some modeling applications. Texture files usually also require generation, in the form of image processing algorithms such as blurring, edge enhancement, etc. Unlike the other rendering resource files, the generation of texture files 206 is not implicit, but requires information from the scene description file 208 as well as texture make parameters 214. Accordingly, texture generation is a highly scene-dependent, and even frame-dependent, task. Upon generation, generated texture files 222 (e.g. tif, jpg, gif, etc.) may then be provided to the rendering engine. The above generation tasks may be performed by the rendering engine itself in a preprocessing task, or may be performed by auxiliary systems prior to job submission.
A problem arises in conventional 3-D computer animation studios today as a result of the massive computational power needed by the rendering engines. There is a trade-off between capital investment in computing resources (in the form of large “rendering farms”) versus the speed at which rendering can happen. For studios large enough to have their own rendering farm, the rendering farm is usually connected to a team of users over a local area network (LAN). Conventionally, rendering jobs are submitted separately by individual users, whose rendering jobs sit in a queue until the rendering engine is available for that job. An administrator (usually a human) often makes real-time decisions about task prioritization among jobs in the queue. According to an exemplary prior art system, generation of rendering resources is not coordinated in a systematic manner, and often the rendering engine and other network resources end up performing redundant generation and/or rendering tasks. The conventional scenario may be particularly frustrating for a user who has only made a minor tweak or addition to their model or another rendering resource. For example, while the user may have simply moved the position of a light, they must again wait in the queue, have their rendering resources generating, and have the rendering process repeated.
Bottlenecks and inefficiencies caused by limited studio computing resources may be remedied somewhat by online rendering services that rent computing time to remote users on a per-session basis. For a price, a remote user may submit their rendering resources (raw or generated) to the online service and, for a greater price, may have their rendering jobs given high priority in the online queue. One shortcoming of conventional online rendering services, however, lies in the massive amount of data that needs to be transferred across the internet. While shader files and procedurals may generally be “slim” resources not requiring excessive bandwidth, scene description files and textures can be massive in size. By way of example, a typical session for rendering frames corresponding to ten seconds of an animated feature would require the transfer of about 2 Gbytes of data for the scene descriptions (raw or generated) and 2 Gbytes of textures. There can be up to 250 Gbytes of textures corresponding to an overall project. Accordingly, it could be quite frustrating for a remote user who has made a minor tweak in their rendering resources to require resubmission of these massive amounts of rendering resource data to the online rendering engine.